As computers and networks progresses rapidly, a variety of kinds of information, such as character data, image data, and audio data, are stored in and transferred between computers and networks. Among this data, a data amount of image data, especially multi-value image data is very large because the data contains much information, and there is a problem that the image data consumes resources such as memory and communication lines when the image data is stored and transferred. To cope with the problem, a high efficiency coding is often used to remove redundancy in an image and/or reduces the data amount by changing the content of the image to a level where the degradation of the image quality is not so conspicuous.
As for high efficiency coding methods, ISO, which is an international standardized coding method for a still image and JPEG recommended by ITU-T, is widely used. JPEG is based on the discrete cosine transform and has a problem of block distortions generated at a high compression ratio.
Resolution of a device in which input or output an image is improved in order to meet the demand for better image quality, a higher compression rate is demanded than ever. Accordingly, a subband coding method, another conversion method, is used. In the subband coding method, a digital image signal is divided into bands by, e.g., a filter for wavelet transform (the band division is called “wavelet transform”, hereinafter), which is then compressed. More specifically, in subband coding, after an inputted signal is filtered by a plurality of filters having different passbands, the filtered signals are down-sampled at intervals corresponding to the passbands, and compressed using a tendency of energy of output signals from the filters (Martin Vetterli, “Wavelet transform and Subband Coding”, Journal of the Institute of Electronics, Information and Communication Engineers, Vol. 74, No. 12, PP. 1275-1278, December 1991).
Subband coding is hierarchical coding, and encoded data can be decoded up to a desired layer of hierarchy without decoding all layers of hierarchy. Subband coding has a virtue of easily realizing efficient decoding, but also has a drawback of including alias which degrade image quality.
Wavelet transform is generally said to be a lower concept or modification of subband coding. It should be noted that “wavelet” in this specification is not limited to a technique using wavelet transform filters, and should be understood to widely include a technique of using filters adapted in subband coding.
FIG. 34 shows a basic configuration of an apparatus which performs band division by filters used in subband coding, e.g., wavelet transform filters, and synthesis. Orthogonal mirror filters are generally used as the wavelet transform filters.
Referring to FIG. 34, reference numeral 1101 denotes an image input processing unit which pre-processes image data; 1102, a discrete wavelet transform unit; 1103, a quantization unit; 1104, an entropy encoder; 1105, a transmission/recording processing unit; 1106, a reception/reproduction processing unit; 1107, an entropy decoder; 1108, an inverse quantization unit; 1109, an inverse discrete wavelet transform unit; and 1110, an image output unit. The configurations from the image input processing unit 1101 through the transmission/recording processing unit 1105 constitute an encoding apparatus, and data outputted from the encoding apparatus is transferred via communication or a recording medium to an decoding apparatus, constituted by the configurations from the reception/reproduction processing unit 1106 through the image output unit 1110, where an image is reproduced.
Encoding processing is explained first. An input image signal undergoes processes, such as γ conversion and color conversion, in the image input processing unit 1101, and enters the discrete wavelet transform unit 1102. The discrete wavelet transform unit 1102 performs discrete wavelet transform on the input image signal both in the vertical and horizontal directions. In wavelet transform processing, wavelet transform is recursively performed on a low frequency component obtained by wavelet transform, thereby generating multiple layers of hierarchy. The discrete-wavelet-transformed image data is inputted to the quantization unit 1103. The quantization unit 1103 can change the compression ratio and coding rate by quantizing the wavelet data with predetermined characteristics. The quantized image data is compressed by the entropy encoder 1104 utilizing the redundancy of the data. The transmission/recording processing unit 1105 performs processes for transferring the coded data, and for converting the coded data to a format suited for a medium when recording the data on the medium. The image is hierarchically encoded in the foregoing processing, and the output data is transferred or recorded on a storage medium.
The decoding processing is realized by performing the processes of the encoding processing in the inverse order. First, the reception/reproduction processing unit 1106 separates and reproduces image data which is transferred or recorded on a medium, and the entropy decoder 1107 decodes the image data, the inverse quantization unit 1108 decodes the image data to data before quantization, then the inverse discrete wavelet transform unit 1109 decodes the data into the original image data. Thereafter, the original image data is converted to image data suitable for designation of output by the image output unit 1110, then outputted. In this processing, it is possible to obtain an image of a desired small size by performing the inverse wavelet transform processing up to an intermediate layer of hierarchy, by taking advantage of the virtue of hierarchical encoding. Since the wavelet transform is generally octave transform, an image, corresponding to layer/layers of hierarchy, whose size is 1/2n of the original size can be easily obtained.
With reference FIGS. 35 and 36, discrete wavelet transform in encoding processing is mainly described in more detail, and with reference to FIG. 37, discrete wavelet transform in decoding processing is mainly described in more detail.
In FIG. 35, the same elements as those in FIG. 34 are referred to by the same reference numerals. Image data is inputted to a low pass filter 1203 for analysis (referred to as “analysis LPF”, hereinafter) for band division and to a high pass filter 1201 for analysis (referred to as “analysis HPF”, hereinafter) for band division, and separated into a low frequency band signal and a high frequency band signal. Since the band width of each separated signal is a half the band width of the original image data, the down samplers 1202 and 1204 sample the signals. The high frequency band signal is directly outputted to the quantization unit 1103. The low frequency band signal is recursively processed by an analysis HPF 1205, an analysis LPF 1207, and down samplers 1206 and 1208. FIG. 35 shows an example of outputting three layers of hierarchy by two cycles of processing for explanatory convenience. Its concept is shown in FIG. 38. By the two dimensional (vertical and horizontal) processing, data corresponding to HL1, LH1, HH1 shown in FIG. 38 is obtained by the analysis HPF 1201 and the down sampler 1202, and data corresponding to LL1 is obtained by the analysis LPF 1203 and the down sampler 1204. However, the LL1 is recursively processed, and data corresponding to HL2, LH2, HH2 in FIG. 38 is obtained by the analysis HPF 1205 and the down sampler 1206, and data corresponding to LL2 in FIG. 38 is obtained by the analysis LPF 1207 and the down sampler 1208.
The respective frequency band data is quantized by the quantizer 1103a to 1103c corresponding to the respective frequency bands, and encoded by the entropy encoder 1104 for each band, then outputted by the transmission/recording processing unit 1105.
FIG. 36 shows an example of another configuration of the discrete wavelet transform unit 1102. In FIG. 36, the same elements as those in FIG. 35 are referred to by the same reference numerals. In the configuration shown in FIG. 36, when data of all the layers of hierarchy of an image is not needed, the transmission/recording processing unit 1105 outputs information indicating it to a controller 1209. In turn, the controller 1209 controls so that data of a needless layer/layers of hierarchy is not encoded out of hierarchically corded data of each frequency band. If high resolution data of a high layer/layers of hierarchy is not needed, the switch unit 1210 opens a switch or switches corresponding to the data of the higher layer/layers of hierarchy, thereby preventing the data of the layer/layers of hierarchy which is not encoded from being inputted to the entropy encoder 1104. At this time, it is possible to reduce a processing load and energy consumption by stopping the needless operations performed by a part of the discrete wavelet transform unit 1102 and the entropy encoder 1104.
In FIG. 37, the same elements as those in FIG. 34 are referred to by the same reference numerals. In the example shown in FIG. 38, an inverse quantizer 108a corresponds to the quantizer 1103a for quantizing hierarchical data of HL1, LH1 and HH1, an inverse quantizer 1108b corresponds to the quantizer 1103b for quantizing hierarchical data of HL2, LH2, and HH2, and an inverse quantizer 1108c corresponds to the quantizer 1103c for quantizing hierarchical data of LL2 in the low frequency band. The inversely quantized signals sequentially undergo inverse wavelet transform from the data of LL2.
First, LL2 is interpolated with 0's by an up sampler 1301, and further undergoes interpolation processing by a LPF 1302 for synthesis (referred to as “synthesis LPF”, hereinafter) for band synthesis. HL2, LH2, and HH2 are similarly up-sampled by the up sampler 1304, undergo interpolation processing by a HPF 1305 for synthesis (referred to as “synthesis HPF”, hereinafter) for band synthesis, then added to the LL2 component by an adder 1303, thereby LL1 is reproduced. The reproduced LL1 is further up-sampled by an up sampler 1306, and interpolated and reproduced by a synthesis LPF 1307. HL1, LH1, and HH1 are also up-sampled by an up sampler 1309, interpolated and reproduced by a synthesis HPF 1310, then added to the LL1 component by an adder 1308, thereby an original image is reproduced.
By selectively outputting data out of the respective hierarchical data acquired in the above processing steps in the image output unit 1110, an image having 1/2n of the original size can be easily outputted.
In the conventional coding method based on subband coding or wavelet transform, a range where a rip occurs becomes wide in the blocking frequency range of a filter when a filter having many taps, such as finite impulse response (FIR) filter is used for band division, and ringing is caused around portions where signal level changes greatly, e.g., at edges in an image. Especially for recursively performing band division a plurality of times, the same filter is conventionally used for all the band division processes. This causes a relative increase in the number of taps of a filter due to down sampling operation as the band division progresses. As a result, ringing occurs in a wide area around an edge of an image. To cope with this problem, a filter having a smaller number of taps and having a broad passband characteristic is used to restrain the occurrence range ringing. As the wavelet transform filter, an orthogonal mirror filter is generally used.
FIG. 39 shows characteristics of a 9×7 filter using the orthogonal mirror filter used in JPEG2000 which is recently suggested. It can be seen that both of the LPF and HPF have broad passband characteristics. However, due to the broad passband characteristics, aliasing occurs when a signal is down sampled after filtering an image signal for band division. However, the aliasing included in the image signals passed through the respective LPF and HPF is theoretically cancelled by synthesizing the filtered image signals without performing quantization. But when an image signal corresponding to an intermediate layer/layers of hierarchy from a LPF is extracted and outputted as an image signal, aliasing is not cancelled because no signal component from a HPF is synthesized.
This problem is caused because a condition of complete reproduction is not satisfied when decoding frequency divided signals, and aliasing generated by the HPF and LPF can not be cancelled to each other.
The effect of the aliasing generated by the LPF appears as ringing near edges where levels largely change. This is because the edges include signals in high frequency band but there is not a high frequency component, and aliasing generated by the LPF can not be cancelled. As a result, a high frequency component even affects a low frequency component as alias which generates noise in an area where the signal level gradually changes in an image. In short, the image is degraded in the low frequency component.
In a scheme of extracting signals up to an intermediate layer of hierarchy, only the LPF signal component which naturally includes alias is extracted and outputted, and therefore, an image of good quality can not be obtained.
The characteristics of the LPF and HPF are defined in the Standard. If a LPF and/or HPF having modified characteristics is used, compatibility is not maintained and moreover, the Standard itself is broken.
There is a method that, when the size of an input image is different from that of the output image, the resolution of an image inputted to an encoding apparatus is changed in advance. In this case, any known resolution conversion means can be used. Since it is necessary to change the settings of, inter alia, a filter in response to an output resolution, a large capacity is needed for memory and/or hardware when a two dimensional filter is used. Furthermore, when a destination device simultaneously requests a plurality of resolutions, it is necessary to prepare a plurality of devices and the devices have to be operated in response to the destination device.
Further, it is also possible to perform color resolution conversion in hierarchical coding. In a case of converting a 4:2:2 image signal to a 4:2:0 image signal, for instance, the conversion is easily realized by cutting the vertical color component or performing an equivalent process. In this case, however, alias can not be cancelled since the high frequency component is cut in the aforesaid conventional method, and an image is degraded due to a residual alias component. Similarly, a 4:1:1: signal can be obtained by cutting a horizontal color component, and a 4:1:0 signal can be obtained by cutting vertical and horizontal color components, but there is the same drawback as set forth above.